DOCUMENT RESUME 



ED 415 639 



EC 306 122 



AUTHOR 

TITLE 



INSTITUTION 



SPONS AGENCY 

PUB DATE 
NOTE 

CONTRACT 
AVAILABLE FROM 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Liu, Kristin; Spicuzza, Richard; Erickson, Ron 
Focus Group Input on Students with Limited English 
Proficiency and Minnesota's Basic Standards Tests. State 
Assessment Series: Minnesota, Report 4. 

National Center on Educational Outcomes, Minneapolis, MN. ; 
Minnesota State Dept, of Children, Families, and Learning, 
St. Paul. 

Office of Educational Research and Improvement (ED) , 
Washington, DC. 

1996-08-00 

22p. 

R279A50011 

National Center on Educational Outcomes, University of 
Minnesota, 350 Elliott Hall, 75 East River Road, 
Minneapolis, MN 55455; phone: 612-626-1530; fax: 
612-624-0879; World Wide Web: http://www.coled.umn.edu/NCEO 
(document may be copied free of charge, additional print 
copies, $5) . 

Reports - Evaluative (142) 

MFOl/PCOl Plus Postage. 

Academic Standards; Accountability; *Achievement Tests; 
♦Educational Assessment; Elementary Secondary Education; 
Evaluation Methods; Focus Groups; Information Needs; 
Language Minorities; *Limited English Speaking; *Minimum 
Competency Testing; Outcomes of Education; State Programs; 
State School District Relationship; Student Evaluation; 
♦Student Participation; Testing Problems 
♦Minnesota; ♦Testing Accommodations (Disabilities) 



ABSTRACT 



This report describes findings of two half-day focus group 
meetings that discussed experiences and concerns about the participation of 
students with Limited English Proficiency (LEP) in Minnesota's Basic 
Standards Exams. The focus groups included test coordinators, general 
classroom teachers, and English as a Second Language teachers. The report 
summarizes the discussions held during the focus groups, including: (1) 

experiences during the Spring 1996 testing, with information on the level of 
participation by students with LEP, how participation decisions were made, 
and provision of accommodations; (2) suggestions for future test 
administrations, including information on interim testing, possible 
accommodations, district decision making processes, temporary exemptions, and 
a Pass -LEP scoring option; (3 ) future district needs , including the need for 
better student information, better test information, and other types of 
tests; and (4) district concerns, including general concerns about setting 
levels of performance, remediation and test preparation, and use of data. 
Possible accommodations discussed included the option to restate directions 
and test questions in different English words to make the test more 
comprehensible to students with LEP, the option to test students with LEP in 
their own classrooms, and the option to use dictionaries on the Basic 
Standards of Written Composition. (CR) 
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The Minnesota Assessment Project is a four-year, federally funded 
effort awarded to the Minnesota Department of Children, Families and 
Learning from the U.S. Department of Education, Office of Educational 
Research and Improvement. The project’s goal is to promote and evalu- 
ate the participation of students with limited English proficiency and 
students with disabilities in Minnesota’s Graduation Standards. 
Specifically, the project will examine ways in which students with lim- 
ited English and students with disabilities can participate in the Basic 
Standards Tests of reading, mathematics, and written composition and 
in the performance-based assessments of the high standards in the 
Profile of Learning. 
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Overview 



Minnesota has joined numerous other states in the call for higher 
standards in elementary and secondary education. Following this 
national trend, it is developing assessment and accountability systems 
that will help ensure that Minnesota’s students are reaching those 
standards. 

In the spring of 1996, Minnesota students participated in the first 
administration of the Basic Standards Tests, one component of 
Minnesota’s new educational assessment system. Although this 
administration of reading and mathematics exams (two of the three 
components of the Basic Standards Tests) was conducted on a voluntary 
basis, most school districts in Minnesota participated in the assessment. 

Besides encouraging districts to participate in the 1996 testing cycle, the 
state of Minnesota also encouraged districts to include aU students who 
would be eligible to participate. Surveys of some of the participating 
districts (see Minnesota Report 2) suggested that districts did, in fact, 
attempt to include students with limited English proficiency (LEP) in the 
assessments. Most districts indicated that more than 50% of their 
students with limited English proficiency participated in the assessment. 
Districts also indicated that they provided some assessment 
accommodations to students, even though in many cases, they did not 
yet have guidelines about the use of accommodations. Furthermore, 
districts indicated the need for several kinds of assistance from the state. 

When the survey was developed, it was considered important to follow- 
up on the information obtained through it by inviting a group of 
individuals to participate in half-day focus groups to discuss experiences 
and concerns in greater depth. This report is a summary of the 
discussions held during the foeus groups. 
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Procedures 



Two half-day focus group meetings were held to follow-up on the 
Spring 1996 testing, with particular focus on students with limited 
English proficiency. Attending the first half-day meeting were the 
testing coordinator, a general classroom teacher, and an English as a 
Second Language (ESL) teacher from each of three metropohtan area 
school districts. For the second half-day meeting, individuals holding 
similar positions in four greater Minnesota school districts attended. 
Participating in the focus group meetings were: 



July 23rd Focus Group for Metropolitan Districts: 

Anoka-Hennepin: Julie Bryant, Ruth Castle, and Gayle 
Walkowiak 

Bloomington: Doris Frace and Bob Peterson 
Minneapolis: Bev Lillquist, David Rathbum, Judy Strohl, and 
Soua Yang 

St. Paul: James Evans, Zhining Qin, and Julieta Fajardo Wahlberg 

July 24th Focus Group for Greater Minnesota Districts: 

Minnesota River Valley Cooperative: Barb Bahson, Cindy Curtis, and 
Amber Volkman 

Mountain Lake: Jim Brandt, Cynthia Celander, and Brenda Feil 
St. Cloud: Judy Germundson, Tom Prescott, and Karmin Schraw 
Willmar: Donna Cairns and Marv Lucas 



These individuals participated in the focus group meetings with interest 
and concern about how students with limited English proficiency could 
best participate in Minnesota’s educational reform efforts. The Project is 
indebted to them for their enthusiastic participation. 

For both of the meetings, the discussion followed an agenda that 
outlined the questions to be covered. To summarize these discussions, 
this report has been organized into four general sections: (1) experiences 
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during the Spring 1996 testing, (2) suggestions for future test 
administrations, (3) future district needs, and (4) future concerns. 



I. Experiences During Spring 1996 Testing 

A. Levels of Participation by Students with Limited 
English Proficiency 

Most participants said that their schools or districts tested the majority of 
LEP students in the spring testing cycle because they wanted to receive 
some basic data on how LEP students were doing. One metropolitan 
district tested all LEP students unless they had been in the U.S. less 
than three years. Another Metropolitan district tested aU eighth graders, 
including LEP students. In a third metropolitan district, any LEP student 
who was willing to take the test was included. The majority of the 
districts representing greater Minnesota tested all of their students 
(except for one district that allowed LEP students to be exempt if they 
had been in the U.S. for under 3 years). In some of the greater 
Minnesota districts there were no eighth graders identified as LEP. In 
other districts, there were students who spoke English as a second 
language but were not categorized as LEP, because they were not being 
served in ESL classes at the time of the spring testing. 



B. How Participation Decisions Were Made 

In the greater Minnesota school districts represented at the meetings, 
decisions about testing were made by small groups of testing technicians 
and others working together. Because students often transfer back and 
forth between small school districts, some participants said that they felt 
the need to have some consistency between districts. Other school 
districts in greater Minnesota said that teachers serving as case managers 
for individual students had made decisions. One Metropolitan district 
allowed parents of LEP students to request an exemption for their child 
by putting the request in writing; they received very few requests. In 
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another Metropolitan district, if the student had been in the district a 
short amount of time, the student’s ESL teacher made the decision about 
whether the student should participate. 

Many participants said that while administrators understood ways to 
include LEP students in the spring testing cycle, the limited time that 
districts had to conduct testing made it difficult for other staff or faculty 
to be as well prepared. In general, several participants agreed that the 
time was too short to make adequate preparations. Participants from 
greater Minnesota said that teachers did not understand the process well 
enough and there was a lot of confusion. 

C. Providing Accommodations 

Most schools and districts represented at the meetings did not provide 
LEP students with accommodations during the Reading and 
Mathematics Standards Tests. A few did allow extra time for LEP 
students, and a separate room for testing. 



II. Suggestions for Future Test Administrations 

A. Interim Testing 

Several participants suggested developing interim tests to be given at 
earlier grades. These tests could have easier items but be aimed at the 
same content standards. The tests could be administered to LEP students 
who are excused from Basic Standards testing through a temporary 
exemption. Interim testing is less frustrating for students because they 
know how far there is to go; it makes it clear what the student needs to 
do. Participants thought that such testing would also be useful for 
accountability purposes, since it can show progress over time. An 
interim test would show that instruction is doing something. 

Participants suggested that funds be made available through the Project 
to provide translated practice tests and other practice materials to help 
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ESL teachers prepare LEP students for the Basic Standards Tests. The 
following issues were raised about practice materials: 

• One participant expressed concern about the legal 
implications of having practice materials available in 
some languages but not in others. It was noted, 
however, that equity concerns were less for test 
preparation than for the testing itself. 

• One metropolitan district is currently developing its 
own practice materials for the Basic Standards Tests. 

One participant said that it would be useful to have a 
central effort to collect and share practice materials 
among the districts. Other participants stated that they 
would appreciate practice materials made from released 
test items when those were available. They would also 
like sample writing prompts for the Basic Standards 
Test in written composition. 

• Some participants believed that test preparation 
materials and sessions should be in English since 
students are expected to do the test in English to pass at 
the designated “Pass-State” level. Others wondered 
whether it would be possible to do test preparation in 
bilingual classrooms during regular class time. 

• The use of regular dictionaries written in various first 
languages of LEP students was mentioned as a crucial 
need by several participants. Mathematics dictionaries 
were also cited as an important part of practice 
materials. 
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B. Possible Accommodations Needed by Students with 
Limited English Proficiency 

When asked about possible accommodations that should be available for 
LEP students in future testing cycles, the following points were raised: 

• Some participants thought that the option to restate 
directions and test questions in different words (but 
still in English) would be useful to help make the test 
more comprehensible to LEP students. One suggestion 
offered was to provide more than one set of test 
directions, with each set worded in a slightly different 
way. This way, if a student asked for clarification of, a 
word or concept in the directions, test administrators 
could still be reading a uniform script, while the 
student would receive a paraphrase in words that he or 
she could understand. Several participants suggested 
that if they could receive test directions ahead of the 
test, ESL teachers could go over them with their 
students, and avoid the need for translating directions 
in many cases. A representative of the Minnesota 
Department of Children, Families and Learning 
reported that non-secure testing materials can be sent 
out in advance. 

• Some participants favored the use of audio tapes on 
which directions and questions would be heard orally. 
(Students in special education had this option in the 
Spring 1996 testing cycle.) It was thought that tapes 
would also be useful for LEP students, but some 
participants questioned whether enough tapes would be 
available for all the students who might need them. 

• Regarding accommodations in setting or timing, one 
participant thought that ESL teachers should be 
allowed to test their own students in their own 
classrooms. However, another participant believed that 
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LEP students should be tested in the same setting as all 
other students, and should not be singled out for 
special treatment. While most participants thought that 
extended time is an important accommodation, it 
should be given in such a way that it does not draw 
attention to the LEP students who need it. If LEP 
students are escorted out of the common testing room 
to another room where they can finish the test, it will 
be obvious to everyone that they are “different.” 

• Some participants favored the option to use dictionaries 
as a possible accommodation on the Basic Standards 
Test of Written Composition, and some did not. One 
participant commented that a dictionary could be 
helpful for some LEP students - that they should be 
given the option to use one if it meant passing the test 
earlier with the dictionary’s help. Some participants felt 
that a dictionary would not help on the test of written 
composition, because students generally use the 
vocabulary that they know and are comfortable with. 
Participants from one metropolitan district said that 
their district has allowed LEP students to use a 
dictionary on writing tests in the past and that this has 
been useful. 

• One participant said that in the case of mathematics, it 
would be helpful to restate the test directions and 
questions in the student’s first language. Other 
participants felt that this wouldn’t be an allowable 
accommodation. 
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C. District Decision Making Processes 

Participants from greater Minnesota districts suggested that an “LEP 
team” make testing decisions for each LEP student in the same way that 
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lEP teams make decisions for students with disabilities. Other group 
members stressed the need for LEP students and parents to know what 
their options are so that they can make informed choices. These 
individuals suggested that decisions about exemption should involve the 
student, his or her parents, administrators, and teachers. While this 
group would make the decision, it could not be an excessively formal, 
complex, or costly process. It was suggested that a third-party, impartial 
advocate for the child might also be a member of this decision making 
team. Some participants mentioned that students might not want to be 
exempted even if they qualify - that they should be given the choice to 
participate. In the metropolitan districts, participants believed that it 
might not be possible to get the parents of each LEP student involved in 
decision making, so the schools or districts may have to set up a parent 
board to review exemptions. 

One participant thought that the types of accommodations allowed 
should be left up to the individual teachers and their LEP students. 
These people know what would be most likely to work best in each 
case. 

D. Temporary Exemptions for Students with Limited Engiish 
Proficiency 

The focus groups discussed the current temporary testing exemption for 
any LEP student who has had three or fewer years of instruction in a 
school where the primary instructional language is English. A common 
concern held by many of the participants was the impact that such an 
exemption might have on the provision of needed services to LEP 
students. Participants from greater Minnesota stated that in some small 
school districts with limited resources, services for LEP students will 
not be provided unless the state educational agency mandates such 
services. Group members considered restrictions on this exemption 
policy. For example, if an exemption was not allowed until the end of a 
student’s senior year, the school district would be more motivated to 
provide services up to that point. 
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On a similar note, a proposed exemption of five years for the Basic 
Standards Test in Written Composition was not considered useful 
because it would make it even easier for districts to avoid serving LEP 
students. Participants thought that a three-year exemption on the writing 
test was the maximum that should be allowed. The general consensus 
was that, as early as possible, students should be required to take the 
tests in order to know how they match up to the standards. The 
participants thought it would be most beneficial to have a common 
exemption policy across all three Basic Standards Tests. They expressed 
that the most important thing is to help the students, and a five-year 
exemption will not do that in the long run. 

A proposal was made that LEP students be required to have their parents 
sign a formal exemption request form. This would add credibility to 
decisions and cause all parties to consider their decisions more carefully. 

E. “Pass-LEP” and “Pass-Translate” Performance Levels 

Overall, participants seemed to be undecided about a “Pass-LEP” 
scoring option for the Basic Standards writing test. They liked the idea 
in principle, but thought that “Pass-LEP” would be problematic and 
could have legal implications. The following points of view were 
expressed: 

• Some participants suggested that different scoring for the 
“Pass-LEP” option would not be feasible due to the difficulty 
in allowing for the different types of errors that are made by 
each language group. How does one allow for errors that 
speakers of an African language might speak, as compared to 
speakers of Hmong? However, other participants pointed out 
that it is usually fairly clear whether a paper is written by a 
non-native speaker of English because there are some standard 
types of errors that native speakers of English do not 
commonly make. For example, allowance could be made for a 
lack of plural endings (-s, -es) on words, since this is a 
common second language acquisition problem for people from 
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many different first languages. Native speakers of English do 
not usually have a problem with plural word endings. 

One metropolitan district representative suggested that LEP 
students might need the “Pass-LEP” option, based on the 
member’s prior experience with district writing tests. LEP 
students in the district were “punished” by scorers for their 
lack of proficiency in Enghsh (i.e., certain grammatical 
structures that are slow to develop because of interference 
from the student’s first language). Another metropohtan 
district representative thought that it might be good to have 
marginal LEP student writing rescored by a trained group of 
ESL teachers who are familiar with typical second language 
acquisition problems. Other participants agreed with this 
suggestion. 

Since the option is not given for the Basic Standards Test in 
reading, some participants thought that all three exams 
(reading, mathematics, and written composition) should be 
treated the same - that no special performance standards for 
LEP students should be allowed. 

Many participants thought that if there is a “Pass-LEP” option, 
it should be reserved as a last resort, perhaps in the spring of 
the senior year, for LEP students who are unable to pass at the 
“State” level. Similar beliefs were expressed toward the “Pass- 
Translation” option, which is to be provided for LEP students 
needing translated versions of the Basic Standards Test in 
mathematics. Participants thought that this designation might 
also be problematic, and raise certain legal issues if it were 
used. 
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III. Future District Needs 



A. Better Student Information 

Participants expressed a desire to code certain types of student 
information directly on test answer forms to use when later examining 
the participation of LEP students in the Basic Standards Tests. 
Currently, some districts are forced to review test records manually to 
find this information, a costly and time consuming process. When asked 
what information they would like to see on an answer sheet, participants 
suggested the following: 

• Some participants wanted to see information about the 
student’s first/home language entered on the answer 
sheet, instead of relying on the test administrators to 
identify students who “appear to be” LEP. There was 
some concern that test administrators who are not 
familiar with LEP students may identify LEP on the 
basis of appearance (e.g., identifying a Korean adoptee 
as LEP because the student is Asian). It was noted that 
home language information can be obtained through 
the student’s Minnesota Advanced Reporting Student 
System (MARSS) tracking number, although some 
participants expressed concern over the efficiency of 
the MARSS system. It was also suggested that ESL 
teachers could be asked to complete the student 
information section, and state whether or not a student 
is receiving LEP services. Some participants from the 
metropolitan districts said that ESL teachers might not 
be willing to give this information. 



• Other participants suggested collecting data from LEP 
students on their length of residence in the district. 
There was disagreement on the issue of whether these 
data should be collected from all students. Some 
participants thought that if all students are asked about 
their length of residence, the information would not be 
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as useful. Others thought that it would be useful to 
have this information on aU students, as a possible 
explanatory factor for schools or districts showing low 
performance on the tests. 

• The National Assessment of Educational Progress 
(“The Nation’s Report Card”) test was cited as an 
example of an assessment that has an excellent student 
data sheet accompanying it. Members suggested 
referring to this for further guidance in collecting 
student information. 




B. Better Information About the Tests 

Participants from greater Minnesota expressed the need for their ESL 
teachers to know more about the testing program in order to effectively 
communicate information to the parents of their LEP students. They 
suggested that the Miimesota Department of Children, Families and 
Learning (CFL) produce a reference paper or brochure with questions 
that regular classroom teachers commonly ask about the testing, along 
with the answers to those questions. In one greater Minnesota district, 
special education teachers made such a brochure for regular classroom 
teachers, and it was successful. A brochure relating to LEP issues in 
testing would be helpful so aU teachers have the same information. 
Participants thought it would be especially important to clarify the terms 
“accommodation” and “modification” in this brochure, and to articulate 
clear policies on temporary testing exemptions, so that the same 
situations are interpreted in the same way by all staff. 

A similar suggestion was made for the CFL to develop a brochure for 
parents and to translate this document into the languages that are most 
dominant in the state. While it was recognized that a videotape would be 
ideal (since not aU parents of LEP students can read in their native 
language) a brochure would certainly be helpful. 



Several group members thought the Department needed to publish a list 
of acceptable accommodations for LEP students so that teachers know 
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what is allowable under the current guidelines. Others expressed a 
desire for the CFL to print a list of what types of skills will be assessed, 
and what content students are required to know. It was thought that 
students and parents have a right to know this and the information 
should be given clearly and concisely. These individuals believed that 
the expectations for the Basic Standards Test in mathematics were fairly 
clear, but not so clear in the reading assessment. 



C. Other Expressed Needs 

• Several participants expressed a desire for some kind 
of testing for LEP students who do not pass or are 
exempted from Basic Standards testing. A low score 
on a language proficiency test would document why a 
student is being exempted from the tests, and would 
give students an idea of where they are in their 
language acquisition. 

• Districts need help on types of remediation to offer 
LEP students who do not pass the tests. For example, 
participants in both sessions did not feel particularly 
well prepared to deal with students who need remedial 
reading instruction. Junior high and high school 
teachers are not commonly trained in direct reading 
instruction. 



• Participants asked for intermediate tests to be 
developed for elementary school as a step in preparing 
students for the Basic Standards Tests. They expressed 
the need for some kind of testing continuum, to show 
students exactly what they need to accomplish, and 
help them work their way up to the necessary level of 
achievement. Reference was made to other states such 
as Oregon and New York, which are currently using 
these types of intermediate tests. 




• A concern was raised that few measures currently exist 

N C E O 
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that clearly define levels of language proficiency, 
especially in reading. Group members asked whether 
the Department could assist in identifying such 
instruments for local districts to use. Others questioned 
whether performance on norm-referenced tests could 
be used to determine adequate progress toward student 
readiness for the Basic Standards Tests. 

• Representatives from districts in greater Minnesota 
asked that the CFL or the University of Minnesota 
provide additional training opportunities for teachers 
and administrators in knowing how to prepare LEP 
students for the Basic Standards Tests, and in making 
participation decisions. 



IV. District Concerns 

A. Setting Levels of Performance 

Some participants expressed concern over the process of “putting 
children into small boxes” with all the different levels of passing. They 
found the different levels confusing and thought that it could be harmful 
to students to have varying standards. For example, participants 
cautioned that the effect of “Exempt” on a student’s transcript could be 
detrimental when that student later applies for employment. If high 
school diplomas are to mean something, it was believed that exemption 
should be very limited. 

One participant believed strongly that LEP students should be given the 
responsibility of learning the skills that are needed to pass the test. This 
group member thought that students should have opportunities to 
practice, but they should not be given special treatment because they will 
not receive special treatment from employers. Another participant added 
that singling out LEP students for special treatment in testing situations 
may violate some cultural ideas about the value of fitting in. If LEP 
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students are singled out, some of the students may “lose face” with 
family or peers. 

Many participants worried about whether their LEP students would be 
able to pass the tests and graduate if the tests are only provided in 
English. They thought that assessment could be helpful if it is framed 
well, but a system that only has pass or fail options turns assessment 
into punishment. 

B. Basic Standards Tests of Reading and Mathematics 

Many participants said that the Basic Standards Test in Reading assesses 
different types of reading skills than they have been teaching. Teachers 
of elementary and junior high often teach reading for pleasure. The 
reading test assesses reading for information, an activity many students 
do not necessarily enjoy, or do extensively in school. One participant 
expressed her wish that the reading selections be geared toward a 
secondary student interest level. 

If the state passes on the responsibilities of making written translations 
of tests (for the “Pass-Translation” option on the mathematics test) to the 
individual districts, districts may have difficulty making those translated 
versions available to all language groups within a district. There is a lack 
of money available for having translations made and a lack of available 
translators. There is a great deal of concern about the legal problems that 
could arise from a situation in which a translated test is not available for 
a student from a particular language group. Participants requested help 
from the CFL and the University of Minnesota in dealing with this 
translation issue. 

Participants questioned the delay of the written composition exam until 
tenth grade. Some members believed that giving the writing test earlier 
would allow more time for remedial efforts aimed at those students who 
did not pass during their initial attempt. 
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C. Remediation and Test Preparation 

Participants from smaller districts raised concern over remediation, 
fearing that regular classroom teachers will be required to 
simultaneously provide instruction in the regular curriculum and 
remediation for students who did not pass the test. Questions were 
raised as to how teachers will deal with students who fail the reading 
test when they are not necessarily trained in direct reading instructional 
techniques. Participants from greater Minnesota stated that they do not 
have the staffing and resources to provide remediation for LEP students 
who fail the exams. 

Many participants expressed concern that too much instructional time 
may be given to preparing for the test, with too much emphasis on using 
old test forms and practice questions. Instead, they believed that 
students should have to come in before or after school to attend test 
preparation sessions. 

D. Use of Data 

Metropolitan area participants questioned how the performance data on 
LEP students would be used once it was available. They stated then- 
desire to encourage all of their students to take the Basic Standards Tests 
and to work for a diploma at the state level, but they were concerned that 
the participation of these students may make a district’s overall scores 
lower. These participants worried that test results could be used in 
harmful ways by the media and others, and that such misuse could have 
a negative impact on their ability to serve LEP students. Members 
believed strongly that the department and state need to clarify the 
reasons why they want the data; in short, is it for accountability or for 
school reform? A suggestion was made that the Project seriously 
consider ways to address the issues of public reporting, so that 
information can be shared with the public in an equitable manner, and 
that people are given an accurate picture of how well students are doing. 
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E. Other Expressed Concerns 

Participants from greater Minnesota with significant minority student 
populations expressed concern about the LEP students who belong to 
the migrant worker population. These students may be present in school 
during the fall and spring, but absent during the winter. If the test is 
given in the winter, what effect will this have on the LEP students who 
do not take the test because they have temporarily moved away? Testing 
dates will have an impact on school district policies for graduation of 
these students. In addition, children of migrant workers do not always 
come to school with current transcripts to show how long they have 
been in school in various places. For this reason, it could be very 
difficult to decide whether they should be exempt from testing. 
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